CHAPTER 10 Having Confidence in Your Results 137

in the earlier section “Before you begin: Formulas for confidence limits in large

samples.” So the SE of the 36 observed serious falls in a three-month period is

simply

36 , which equals 6.0. So for the confidence limits, we have

CLL

36 0

1 96

6 0

.

.

. and CLU = 36.0 + 1.96 × 6.0. In this case, the ME is 11.76,

which works out to a 95 percent CI of 24.2 to 47.8 serious falls in the three-

month period.

Many other approximate formulas for CIs around observed event counts and rates

are available, most of which are more reliable when your N is small. These formu-

las are too complicated to attempt by hand, but fortunately, many statistical pack-

ages can do these calculations for you. Your best bet is to get the name of the

formula, and then look in the documentation for the statistical software you’re

using to see if it supports a command for that particular CI formula.

Relating Confidence Intervals

and Significance Testing

In Chapter 3, we introduce the concepts and terminology of significance testing,

and in Chapters 11 through 14, we describe specific significance tests. If you read

these chapters, you may have come to the correct conclusion that it is possible to

assess statistical significance by using CIs. To do this, you first select a number

that measures the amount of effect for which you are testing (known as the effect

size). This effect size can be the difference between two means or the difference

between two proportions. The effect size can also be a ratio, such as the ratio of

two means, or other ratios that provide a comparison, such as an odds ratio, a

relative risk ratio, or a hazard ratio (to name a few). The complete absence of any

effect corresponds to a difference of 0, or a ratio of 1, so we call these the “no-

effect” values.

The following statements are always true:»

» ^{If the 95 percent CI around the observed effect size includes the no-effect value,}

then the effect is not statistically significant. This means that if the 95 percent

CI of a difference includes 0 or of a ratio includes 1, the difference is not large

enough to be statistically significant at α = 0.05, and we fail to reject the null.»

» ^{If the 95 percent CI around the observed effect size does not include the}

no-effect value, then the effect is statistically significant. This means that if the

95 percent CI of a difference is entirely above or entirely below 0, or is entirely

above or entirely below 1 with respect to a ratio, the difference is statistically

significant at α = 0.05, and we reject the null.